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Projects  this  year 

Detection  and  Discrimination  of  Relative  motion  (D.A.  Silverstein  S.A.  Klein  to  be  submitted) 

We  measured  the  temporal  dynamics  of  relative  rrx)tion  judgement  Two  lines  or  dots  oscillated  either  in 
the  same  phase  of  motion,  or  in  180  phase  relative  to  each  other.  Subjects  were  required  to  discriminate 
between  Ae  two  kinds  of  motion.  The  dots  either  oscillated  vertically  or  horizontally.  The  former  case 
produced  oscillating  orientation  and  the  later  case  produced  oscillating  separation.  The  temporal  frequency  of 
oscillation  was  varied  (presented  at  OSA-1992).  In  experiments  with  lines,  the  separation  of  the  lines  was 
varied  while  the  distance  of  the  lines  from  the  fovea  was  held  constant.  The  discriminability  of  two  different 
phases  of  relative  motion  was  measured  and  compared  to  the  detectability  of  these  motions.  The  psychometric 
transducer  functions  were  measured  and  plonecL  (Presented  ARVO-1993) 

Relevance  of  tiiiman' vision  to  JPEG-DCT  compression  (S.A.  Klein,  D.A.  Silverstein,  T.  Carney) 

Vision  researchers  can  and  should  make  important  contributions  to  the  engineer  dominated  field  of 
image  compresaon.'  An  overview  of  the  field  from  the  point  of  view  of  a  vision  researcher  is  presented  and 
sevCTal  areas,  for. potential  scientific  contribution  are  outlined.  Presented  and  published  in  SPEE,  1992. 

A  radial  pattern  for  demonstrating  MPEG  artifacts  (D..A.  Silverstein,  S.  A.  Klein.  Submitted  with 
grant  proposal  to  AFOSR) 

We  employed  a  simple  test  pattern  to  test  several  aspects  of  MPEG  compression.  The  test  sequence  was 
compressed  and  decompressed  using  the  Stanford  Portable  Research  Group’s  MPEG  codec  (PRVG-MPEG). 
The  availability  of  the  source  code  for  this  MPEG  compressor  allows  us  to  have  complete  control  over  the 
processing  steps  and  to  obtain' intermediate  values  in  the  computations.' Arotating  radial  square  wave  grating 
was  used  to  induce  several  kinds  of  processing  artifacts.  This  test  sequence  was  chosen  to  have  a  variety  of 
continuously  varying  properties.  The  spatial  frequency  varies  from  high  at  the  center  to  low  toward  the  edge. 
Eight  different  angles  of  ^ges'  iii  both  polarities  are  shown  in  each  frame.  Thevelocity  of  the  edge  also  varies 
continuously  from  the  center  to  the  periphery  of  the  fig^- 

Three  experiments  were  conducted.  First,  we  compEessed  the  15  frame  rotary  sequence  described 
previously  and  watched  for  artifacts  introduced  from  the  codec.  The  noticeable  distortions  were  the  same  as 
we  have  seen  in  JPEG  compressed  images.  There  were  small  patches  of  low-contrast,  high-frequency  noise 
around  the  edges  of  the  sectors.  This  type  of  error  occurs  when  the  high  frequency  terms  of  the  DCT  are 
excessively  quantize. 

In  our  second  experiment,  a  frame  of  spatial  white  noise  was  introduced  in  the  middle  of  the  sequence. 
Each  pixel  value  was  chosen  at  random  from  256  gray  levels.  Since  the  frame  immediately  after  the  noise 
frame  could  not  be  predicted  from  the  noise  frame  and  since  MPEG  streams  have  limited  bandwidth,  we  might 
expect  that  the  frame  after  the  noise  friame  would  be  more  severely  degraded  than  the  other  frames.  This  was 
not  the  case,  however,  since  the  frame  after  the  noise  frmne  could  be  backwards  predicted  from  the  next 
frame. 

In  the  third  experiment,  we  limited  the  possibility  for  backwards  prediction  by  sandwiching  two  frames 
of  the  test  pattern  between  two  frames  of  white  noise.  This  image  was  substantially  more  distorted  than  the 
previous  image.  Further,  the  types  of  distortion  are  different.  In  the  image,  16x16  pixel  blocks  (the  size  of 
the  motion  pi^ction  block)  of  high  frequency  noise  can  be  seen,  particularly  in  high  frequency  regions  of  the 
image. 

From  these  experiments  we  can  see  that  perceptual  errors  are  confined  to  small  spatial  regions.  If  these 
areas  of  severe  errors  could  be  automatically  detected,  the  codec  could  be  improved  by  changing  the  bit 
allocation  to  provide  more  information  in  the  severely  ^storted  areas  and  less  information  in  the  areas  that 
were  distorted  below  threshold.  Further,  by  using  a  simple  test  pattern,  it  is  easy  to  see  several  distinct  types 
of  distortions.  Other  simple  test  patterns  will  doubtlessly  reve^  other  types  of  distortions.  Quantifying  and 
cataloging  the  different  types  of  distortions  will  facilitate  their  automatic  detection  and  correction. 

A  DCT  image  Hdelity  metric  and  its  application  to  a  text  based  scheme  for  image 
representation.  (Dj\.  Silverstein,  S.  A.  Klein.  Published  Proceedings  of  the  SPIE  1993) 

For  some  types  of  image  fidelity  measurements,  it  is  useful  to  have  a  very  fast,  spatially  local 
measurement  scheme.  The  discrete  cosine  transform  (DCT)  can  be  used  to  transform  two  images  into  a  space 
where  it  is  veiy  easy  to  obtain  an  estimate  of  the  perceptud  distance  between  two  images,  that  is,  how  many 
times  threshold  the  two  images  are  apart. 

The  DCTT  can  also  be  localiz^,  which  allows  a  piece-wise  comparison  of  the  two  images.  We  used  this 
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algorithm  to  find  the  closest  fit  of  an  ASCII  character  (which  includes  the  english  alphabet,  numbers, 
punctuation  and  a  few  common  symbols)  to  segments  of  an  image.  When  a  digitized  grey-scale  image  is 
segmented  into  many  rectangular  regions  and  each  region  is  matched  with  an  ASCH  character,  a  page  of  text  is 
generated  that  resembles  the  original  image. 

The  text-image  is  in  a  format  that  can  be  displayed  on  a  non-graphic  terminal  and  can  be  sent  via 
electronic  mail  with  the  advantage  that  no  further  processing  is  required  by  the  receiver.  In  addition  to  image 
transmission,  this  processing  scheme  can  be  used  to  preview  stored  images  with  a  text  terminal.  For  example, 
if  you  were  not  sure  if  a  computer  file  contained  an  image  you  wanted  to  send  to  a  printer,  you  could  first  view 
the  image  processed  into  ASCII  characters  on  your  text  terminal.  Since  the  dgorithm  is  computational 
inexpensive,  the  preview  process  takes  very  little  time. 

Restoration  of  compressed  images  (D.A.  Silverstein,  S A.  Klein  To  be  presented  SPIE  Feb  93) 

An  image  that  has  been  compressed  and  decompressed  (cooec)  by  a  lossy  method  becomes  distorted. 
The  distortion  is  usually  manifest  in  some  characteristic  artifact.  For  example,  the  JPEG  compression 
algorithm  often  introduces  a  high-frequency  ’ringing’  artifact  around  high  contrast  edges.  We  have  developed 
a  processing  algorithm  that  can  eliminate  this  artifact  in  many  instances. 

Our  edge  restoring  method  works  by  first  locating  the  edge.  A  Sobel  transform  is  used  to  find  the  area 
around  the  edge.  The  Sobel  transform’s  output  is  filtered  by  first  assuming  a  continuity  constraint  then  by 
thresholding.  This  eliminates  the  spurious  edges  due  to  the  ringing  artifact.  The  remaining  data  is  fit  with 
either  a  straight  line  or  a  curve  by  a  least  squares  method.  The  image  is  then  reconstructed  by  low  pass  filtering 
on  either  side  of  the  fit  edge. 

This  algorithm,  or  any  other  enhancement  algorithm,  cannot  work  in  all  cases.  What  is  noise  in  one 
image  may  be  important  data  in  another.  Fortunately,  in  image  compression,  we  have  an  advantage.  Before  the 
image  is  stored  or  transmitted,  we  have  access  to  the  origin^  and  the  distorted  versions.  The  distoned  version 
is  processed  by  our  edge  enhancement  algCMrithm  and  compared  to  the  original.  For  every  8x8  pixel  block,  it  is 
determined  whether  the  processing  reduces  the  distortion.  This  determination  is  either  made  by  a  model  of 
human  vision  or,  in  our  case,  directly  by  a  human.  For  each  block,  one  additional  bit  is  transmitted  with  the 
compressed  image,  indicating  whether  or  not  the  receiver  should  apply  the  enhancement  algorithm  to  that 
block. 

The  bit  is  included  in  the  parity  of  the  image  block  itself,  so  a  standard  decoder  will  be  able  to  decode 
the  image.  A  decompresser  that  knows  about  the  parity  code  can  generate  a  superior  image  by  applying  an 
enhancement  algoridim  to  the  blocks  that  have  been  labeled  for  post  processing.  Any  other  post  processing 
algorithm  may  be  used  instead  of,  or  in  addition  to,  our  edge  enhancement  algorithm.  The  procedure  may  be 
automated  when  an  appropriate  model  of  human  vision  is  developed,  but  currently  is  useful  in  applications 
where  the  cost  of  a  human  operator  is  not  prohibitive. 


